Dynomotion

Group: DynoMotion Message: 12124 From: Hardy Family Date: 8/17/2015
Subject: Correct way to flash firmware and user programs
Hi Tom,

I read through the previous threads on flashing new firmware etc.  I am in the process of adding some Python class methods to do this for our application, but have a few questions:

We have a continuously running 'supervisor' thread (7) which also has a UserCallback.  Obviously, when flashing new firmware we must be careful to stop that thread cleanly, then after rebooting with the new firmware, it must be recompiled and downloaded to be compatible with the new f/w.

Currently we have a rather laborious process to do this manually using kmotion.exe.  This is a rough outline of what we currently do, but I would like to streamline this for the automated updater.

1. Run a special thread program to zero out the UserCallback, so that nothing goes haywire.
2. Halt the supervisor thread (7)
3. Turn off the "automatically start thread 7" switch (since don't want it to start again on reboot because it might be incompatible with new f/w).
3a: Because I don't know if step 3 is preserved by the next step, we flash user programs JIC.
4. Flash the new kflop firmware.
5. Reboot the kflop.
6. Download new supervisor to thread 7.
7. Check the 'auto start thread 7' switch.
8. Flash user programs.
9. Start thread 7 and continue happily.

So the questions are:

Which of the above steps are not really required?

Does the 'SetStartupThread' script command get preserved to non-volatile memory straight away, or does it need a flash command?

Could we download the new firmware, then our user programs, and flash both before the reboot step?  (If so, we could probably avoid worrying about the SetStartupThread stuff.)  In other words, can we do 'ProgFlashImage' followed by 'Flash' then 'Reboot!'?

Regards,
SJH


Group: DynoMotion Message: 12125 From: Tom Kerekes Date: 8/18/2015
Subject: Re: Correct way to flash firmware and user programs
Hi SJH,

Flashing "New Version" is only flashing firmware to the Flash memory.  Flash memory is never directly executed.  On the next boot this will be moved to Internal and SDRAM Memory and executed.

When "CoffLoad" with pack to flash option for the New Version, the data is temporarily placed in the gather buffer in a format ready to be flashed.  It also doesn't effect the currently running firmware.

So there is no need to stop Thread 7 and to worry about incompatibility.

Actually if the script command is issued to Flash the downloaded New Version the other threads will be temporarily stopped while Flashing to speed up the Flash process but they will resume after Flashing is complete,  But as stated the New Version will not be executed until the next boot.

After the next boot, the New Version will be executing with no options to Launch User Programs on Startup and there will be no User Programs Present.

You might then do your steps 6-9

HTH
Regards
TK


Group: DynoMotion Message: 12126 From: Hardy Family Date: 8/18/2015
Subject: Re: Correct way to flash firmware and user programs
Yes, that helps, thanks.

So to summarize, an efficient way to update firmware and user programs would be:

1. Shut down any running threads cleanly (this is very fast, so may as well do it).
2. CoffLoad(...pack_to_flash) for the new firmware -> gather buffer
3. ProgFlashImage: gather buffer -> flash.  Old firmware still executing in RAM, so safe to go back to step 2 if any errors.
4. Reboot!: boot kflop into new firmware.  No auto thread start, regardless of previous SetStartupThread settings.
4a: Version: read back new version info to confirm booted OK.  If not, admit defeat and start kmotion.exe to recover.
5. CoffLoad(user thread(s)) for updated software -> thread RAM
6. SetStartupThread as required.
7. FLASH: thread RAM -> flash.  Can repeat from step 4 if errors.
8: optionally write config data to high half of 2M flash, using a temporary user thread program.
9. Start thread 7 and continue.

I'll give it a try and report back if any problems.

Regards,
SJH


On Tue, Aug 18, 2015 at 2:38 PM, Tom Kerekes tk@... [DynoMotion] <DynoMotion@yahoogroups.com> wrote:
 

Hi SJH,

Flashing "New Version" is only flashing firmware to the Flash memory.  Flash memory is never directly executed.  On the next boot this will be moved to Internal and SDRAM Memory and executed.

When "CoffLoad" with pack to flash option for the New Version, the data is temporarily placed in the gather buffer in a format ready to be flashed.  It also doesn't effect the currently running firmware.

So there is no need to stop Thread 7 and to worry about incompatibility.

Actually if the script command is issued to Flash the downloaded New Version the other threads will be temporarily stopped while Flashing to speed up the Flash process but they will resume after Flashing is complete,  But as stated the New Version will not be executed until the next boot.

After the next boot, the New Version will be executing with no options to Launch User Programs on Startup and there will be no User Programs Present.

You might then do your steps 6-9

HTH
Regards
TK


Group: DynoMotion Message: 12127 From: Hardy Family Date: 8/18/2015
Subject: Re: Correct way to flash firmware and user programs
One further related question:

Does the top half of the 2M user flash get erased or altered when either ProgFlashImage or user Flash script commands are run? 

Regards,
SJH


On Tue, Aug 18, 2015 at 6:56 PM, Hardy Family <hardy.woodland.cypress@...> wrote:
Yes, that helps, thanks.

So to summarize, an efficient way to update firmware and user programs would be:

1. Shut down any running threads cleanly (this is very fast, so may as well do it).
2. CoffLoad(...pack_to_flash) for the new firmware -> gather buffer
3. ProgFlashImage: gather buffer -> flash.  Old firmware still executing in RAM, so safe to go back to step 2 if any errors.
4. Reboot!: boot kflop into new firmware.  No auto thread start, regardless of previous SetStartupThread settings.
4a: Version: read back new version info to confirm booted OK.  If not, admit defeat and start kmotion.exe to recover.
5. CoffLoad(user thread(s)) for updated software -> thread RAM
6. SetStartupThread as required.
7. FLASH: thread RAM -> flash.  Can repeat from step 4 if errors.
8: optionally write config data to high half of 2M flash, using a temporary user thread program.
9. Start thread 7 and continue.

I'll give it a try and report back if any problems.

Regards,
SJH


On Tue, Aug 18, 2015 at 2:38 PM, Tom Kerekes tk@... [DynoMotion] <DynoMotion@yahoogroups.com> wrote:
 

Hi SJH,

Flashing "New Version" is only flashing firmware to the Flash memory.  Flash memory is never directly executed.  On the next boot this will be moved to Internal and SDRAM Memory and executed.

When "CoffLoad" with pack to flash option for the New Version, the data is temporarily placed in the gather buffer in a format ready to be flashed.  It also doesn't effect the currently running firmware.

So there is no need to stop Thread 7 and to worry about incompatibility.

Actually if the script command is issued to Flash the downloaded New Version the other threads will be temporarily stopped while Flashing to speed up the Flash process but they will resume after Flashing is complete,  But as stated the New Version will not be executed until the next boot.

After the next boot, the New Version will be executing with no options to Launch User Programs on Startup and there will be no User Programs Present.

You might then do your steps 6-9

HTH
Regards
TK


Group: DynoMotion Message: 12134 From: Hardy Family Date: 8/20/2015
Subject: Re: Correct way to flash firmware and user programs
OK, that's excellent that the top half is preserved.

It's getting there, but I am having some problems with the 'flash' command.  Not sure how to wait for it to complete.  After the WriteLine("flash") call, I try using CheckForReady(), but the latter is causing stack corruption in KmotionServer.  Maybe that's because there is a bug in the Linux version, but I would still like to know what the official procedure is for the Flash command (e.g. what sequence of KMotionDLL calls to use).

Regards,
SJH


On Tue, Aug 18, 2015 at 10:20 PM, Hardy Family <hardy.woodland.cypress@...> wrote:
One further related question:

Does the top half of the 2M user flash get erased or altered when either ProgFlashImage or user Flash script commands are run? 

Regards,
SJH


On Tue, Aug 18, 2015 at 6:56 PM, Hardy Family <hardy.woodland.cypress@...> wrote:
Yes, that helps, thanks.

So to summarize, an efficient way to update firmware and user programs would be:

1. Shut down any running threads cleanly (this is very fast, so may as well do it).
2. CoffLoad(...pack_to_flash) for the new firmware -> gather buffer
3. ProgFlashImage: gather buffer -> flash.  Old firmware still executing in RAM, so safe to go back to step 2 if any errors.
4. Reboot!: boot kflop into new firmware.  No auto thread start, regardless of previous SetStartupThread settings.
4a: Version: read back new version info to confirm booted OK.  If not, admit defeat and start kmotion.exe to recover.
5. CoffLoad(user thread(s)) for updated software -> thread RAM
6. SetStartupThread as required.
7. FLASH: thread RAM -> flash.  Can repeat from step 4 if errors.
8: optionally write config data to high half of 2M flash, using a temporary user thread program.
9. Start thread 7 and continue.

I'll give it a try and report back if any problems.

Regards,
SJH


On Tue, Aug 18, 2015 at 2:38 PM, Tom Kerekes tk@... [DynoMotion] <DynoMotion@yahoogroups.com> wrote:
 

Hi SJH,

Flashing "New Version" is only flashing firmware to the Flash memory.  Flash memory is never directly executed.  On the next boot this will be moved to Internal and SDRAM Memory and executed.

When "CoffLoad" with pack to flash option for the New Version, the data is temporarily placed in the gather buffer in a format ready to be flashed.  It also doesn't effect the currently running firmware.

So there is no need to stop Thread 7 and to worry about incompatibility.

Actually if the script command is issued to Flash the downloaded New Version the other threads will be temporarily stopped while Flashing to speed up the Flash process but they will resume after Flashing is complete,  But as stated the New Version will not be executed until the next boot.

After the next boot, the New Version will be executing with no options to Launch User Programs on Startup and there will be no User Programs Present.

You might then do your steps 6-9

HTH
Regards
TK


Group: DynoMotion Message: 12135 From: Tom Kerekes Date: 8/21/2015
Subject: Re: Correct way to flash firmware and user programs
Hi SJH,

I'm not sure what would cause stack corruption.

"Flash" is one of the few KFLOP commands that block for several seconds.  Maybe that is the issue?

The "Flash" command doesn't normally return any response.  The normal "WriteLine" function sends script commands with a "no echo" escape code to KFLOP.  In this case the script command is not echoed back to the PC and KFLOP doesn't send the Ready Prompt after completing the command.  There is a WriteLineWithEcho function that will echo the command and a Ready prompt when completed.  Please try that.

Regards
TK

Group: DynoMotion Message: 12136 From: Hardy Family Date: 8/21/2015
Subject: Re: Correct way to flash firmware and user programs
OK.  One problem I saw when examining the syslog (which we use on Linux) is that I am not grabbing and releasing the lock around the Flash command, so my app continues to poll (GetStatus).  Since the timeout for GetStatus is set to 5s, it manages to fit in a few of these by the time the Flash command is complete.  The kflop/USB is queuing up these commands, and they seem to get executed after flash completes.  The results are coming back, but the DLL is confused and treats these orphan responses as console messages.  I need to look into this, but it seems this might be triggering some bad behavior in the DLL.  For example, there are lines of code like

    response[strlen(response)-2]=0;  // remove the /r /n

when there is no check that strlen(response) >= 2.  In this case, it might indeed trash the stack.

Anyway, I will try the echo version with a timeout suitable for flashing (40sec).  Also grab the token around the call.  Presumably I can then use CheckForReady() to poll for completion.  Any status polling should get rejected by not getting the token.

Regards,
SJH


On Fri, Aug 21, 2015 at 9:56 AM, Tom Kerekes tk@... [DynoMotion] <DynoMotion@yahoogroups.com> wrote:
 

Hi SJH,

I'm not sure what would cause stack corruption.

"Flash" is one of the few KFLOP commands that block for several seconds.  Maybe that is the issue?

The "Flash" command doesn't normally return any response.  The normal "WriteLine" function sends script commands with a "no echo" escape code to KFLOP.  In this case the script command is not echoed back to the PC and KFLOP doesn't send the Ready Prompt after completing the command.  There is a WriteLineWithEcho function that will echo the command and a Ready prompt when completed.  Please try that.

Regards
TK

Group: DynoMotion Message: 12137 From: Hardy Family Date: 8/21/2015
Subject: Re: Correct way to flash firmware and user programs
That seems to work.  Here's the relevant part from syslog:

Aug 21 12:55:59 pluto KMotionServer[28303]: GetAnswerToRequest ENUM_KMotionLock KMOTION_LOCKED
Aug 21 12:55:59 pluto KMotionServer[28303]: GetAnswerToRequest ENUM_WriteLineWithEcho #012#011request=Flash #012#011result=0
Aug 21 12:56:00 pluto KMotionServer[28303]: Console message: Flash#015
Aug 21 12:56:00 pluto KMotionServer[28303]: GetAnswerToRequest ENUM_CheckForReady 1
Aug 21 12:56:13  KMotionServer[28303]: last message repeated 11 times
Aug 21 12:56:13 pluto KMotionServer[28303]: Console message: Program Flash Image Complete#015
Aug 21 12:56:13 pluto KMotionServer[28303]: GetAnswerToRequest ENUM_CheckForReady 1
Aug 21 12:56:14 pluto KMotionServer[28303]: GetAnswerToRequest ENUM_CheckForReady 1
Aug 21 12:56:15 pluto KMotionServer[28303]: Console message: KFLOP 4.33m#015
Aug 21 12:56:15 pluto KMotionServer[28303]: Console message: Ready#015
Aug 21 12:56:15 pluto KMotionServer[28303]: GetAnswerToRequest ENUM_CheckForReady 2
Aug 21 12:56:15 pluto KMotionServer[28303]: GetAnswerToRequest ENUM_ReleaseToken

Note that the echo to the Flash command comes about 1 second after the command.  It then responds with the "program flash image complete" message after 13 seconds, then the Ready response a few seconds later.  (The #015 is octal for the \r character).

For some reason, I have to turn off regular polling (GetStatus); even though I have the lock around Flash, it seems to somehow garble things up.

Also, the CheckForReady() calls are returning KMOTION_TIMEOUT (1) whereas I expected them to return KMOTION_OK (0) when there is no waiting data.  In KMotionIO.cpp, the code in CheckForReady() starts with a test that NumberBytesAvailToRead() indicates at least 1 byte available.  But for some reason that is always true, which is why it is returning a timeout when it tries to read the data.  Maybe there is a difference between the FTDI drivers on Linux and Windows.

Regards,
SJH


On Fri, Aug 21, 2015 at 11:01 AM, Hardy Family <hardy.woodland.cypress@...> wrote:
OK.  One problem I saw when examining the syslog (which we use on Linux) is that I am not grabbing and releasing the lock around the Flash command, so my app continues to poll (GetStatus).  Since the timeout for GetStatus is set to 5s, it manages to fit in a few of these by the time the Flash command is complete.  The kflop/USB is queuing up these commands, and they seem to get executed after flash completes.  The results are coming back, but the DLL is confused and treats these orphan responses as console messages.  I need to look into this, but it seems this might be triggering some bad behavior in the DLL.  For example, there are lines of code like

    response[strlen(response)-2]=0;  // remove the /r /n

when there is no check that strlen(response) >= 2.  In this case, it might indeed trash the stack.

Anyway, I will try the echo version with a timeout suitable for flashing (40sec).  Also grab the token around the call.  Presumably I can then use CheckForReady() to poll for completion.  Any status polling should get rejected by not getting the token.

Regards,
SJH


On Fri, Aug 21, 2015 at 9:56 AM, Tom Kerekes tk@... [DynoMotion] <DynoMotion@yahoogroups.com> wrote:
 

Hi SJH,

I'm not sure what would cause stack corruption.

"Flash" is one of the few KFLOP commands that block for several seconds.  Maybe that is the issue?

The "Flash" command doesn't normally return any response.  The normal "WriteLine" function sends script commands with a "no echo" escape code to KFLOP.  In this case the script command is not echoed back to the PC and KFLOP doesn't send the Ready Prompt after completing the command.  There is a WriteLineWithEcho function that will echo the command and a Ready prompt when completed.  Please try that.

Regards
TK